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Additional  Copies 


To  obtain  additional  copies  of  this  audit  report,  please  contact  the  Secondary  Reports 
Distribution  Unit  of  the  Analysis,  Planning,  and  Technical  Support  Directorate  at 
(703)  604-8937  (DSN  664-8937)  or  FAX  (703)  604-8932. 

Suggestions  for  Future  Audits 

To  suggest  ideas  for  or  to  request  future  audits,  contact  the  Planning  and 
Coordination  Branch  of  the  Analysis,  Planning,  and  Technical  Support  Directorate 
at  (703)  604-8939  (DSN  664-8939)  or  FAX  (703)  604-8932.  Ideas  and  requests 
can  also  be  mailed  to: 

OAIG-AUD  (ATTN:  APTS  Audit  Suggestions) 

Inspector  General,  Department  of  Defense 
400  Army  Navy  Drive  (Room  801) 

Arlington,  Virginia  22202-2884 

Defense  Hotline 

To  report  fraud,  waste,  or  abuse,  please  contact  the  Defense  Hotline  by  calling 
(800)  424-9098;  by  sending  an  electronic  message  to  Hotline@DODIG.OSD.MIL; 
or  by  writing  to  the  Defense  Hotline,  The  Pentagon,  Washington,  D.C.  20301-1900. 
We  fully  protect  the  identity  of  each  writer  and  caller. 
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SUBJECT:  Audit  of  Functional  and  Physical  Configuration  Audits  of  Defense  Systems 
(Project  No.  5AE-0032.05) 


Introduction 

We  are  providing  this  audit  report  for  your  information  and  use.  This  report 
summarizes  our  overall  evaluation  of  the  adequacy  of  the  functional  and 
physical  configuration  audit  process  for  the  acquisition  of  Defense  systems. 
This  report  is  the  sixth  and  final  in  a  series  of  reports  resulting  from  our  audit  of 
functional  and  physical  configuration  audits  of  Defense  systems.  A  functional 
configuration  audit  is  a  formal  examination  of  functional  characteristics  of  test 
data  for  configuration  items  to  verify  that  the  items  have  achieved  their  specified 
performance.  A  physical  configuration  audit  is  a  formal  examination  to  verify 
that  the  configuration  items  "as  built"  conform  to  the  technical  documentation 
that  defines  the  items.  Enclosure  2  provides  definitions  of  technical  terms  used 
in  this  report. 


Audit  Results 

The  Military  Departments'  policies  and  procedures  for  managing  the  functional 
and  physical  configuration  audit  process  were  adequate.  We  identified  no 
systemic  deficiencies  in  the  effectiveness  of  the  functional  and  physical 
configuration  audit  process  for  the  five  Defense  systems  (the  Army  Kiowa 
Warrior  and  Paladin  systems,  the  Navy  Tactical  Command  System-Afloat 
[NTCS-A],  and  the  Air  Force  Navigation  System  Time  and  Ranging  Global 
Positioning  System  [NAVSTAR  GPS]  and  Rapid  Execution  and  Combat 
Targeting  [REACT]  system)  in  our  review.  However,  we  did  identify  areas 
where  the  program  offices  for  the  five  systems  could  individually  improve  their 
management  of  the  functional  and  physical  configuration  audit  process.  The 
results  of  our  review  of  the  five  systems,  including  applicable  findings, 
recommendations,  and  management  comments,  are  synopsized  in  Enclosure  3. 


Audit  Objective 

The  audit  objective  was  to  evaluate  the  adequacy  of  the  functional  and  physical 
configuration  audit  process  for  the  acquisition  of  Defense  systems.  Specifically, 
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we  determined  whether  functional  and  physical  configuration  audits  verified  and 
documented  that  configuration  items  agreed  with  their  configuration 
identifications,  were  complete  and  accurate,  and  satisfied  program  requirements. 
We  also  evaluated  the  management  control  program  as  it  related  to  our  audit 
objective.  In  Enclosure  1,  we  discuss  the  scope  and  methodology  used  to 
accomplish  the  objective  as  well  as  management  controls  and  prior  audit 
coverage. 


Audit  Background 

When  we  began  our  audit,  DoD  Instruction  5000.2,  part  9,  section  A, 
"Configuration  Management,"  contained  the  functional  and  physical 
configuration  guidance  for  the  five  Defense  systems  in  our  review.  The 
instruction  required  program  managers  to  establish  an  effective  configuration 
management  program  to  implement  the  decisions  made  in  the  systems 
engineering  process  by  identifying,  documenting,  and  verifying  the  functional 
and  physical  characteristics  of  a  configuration  item;  controlling  changes  to  an 
item  and  its  documentation;  recording  die  configuration  of  actual  items;  auditing 
the  configuration  item  and  its  configuration  identification;  and  providing  a 
configuration  status  accounting  to  track  changes  in  configuration  items.  During 
the  audit,  the  Deputy  Secretary  of  Defense  issued  DoD  Regulation  5000. 2-R, 
"Mandatory  Procedures  for  Major  Defense  Acquisition  Programs  (MDAPS)  and 
Major  Automated  Information  System  (MAIS)  Acquisition  Programs," 
March  15,  1996.  The  regulation  establishes  requirements  for  configuration 
management.  Subpart  4.3,  "Systems  Engineering,"  requires  that  the  program 
manager  use  a  systems  engineering  process  that  includes  configuration 
management  to  control  the  system  products,  processes,  and  related 
documentation.  Further,  DoD  Regulation  5000.2-R  mandates  that,  as  part  of 
systems  engineering,  the  program  manager  should  establish  a  configuration 
management  process  to  identify,  document,  and  verify  the  functional  and 
physical  characteristics  of  an  item;  record  the  configuration  of  an  item;  control 
changes  to  an  item  and  its  documentation;  and  provide  a  complete  audit  trail  of 
decisions  and  design  modifications.  As  a  result  of  the  new  guidance,  we 
tailored  our  review  to  conform  with  the  guidance  in  DoD  Regulation  5000.2-R. 


Discussion 

Overall,  the  Military  Departments'  policies  and  procedures  were  adequate  for 
managing  the  functional  and  physical  configuration  audit  process  and  tailoring 
their  configuration  management  processes  to  comply  with  DoD  Regulation 
5000.2-R.  We  based  our  audit  of  management  of  the  functional  and  physical 
configuration  audit  process  on  the  evaluation  of  five  Defense  systems:  the 
Kiowa  Warrior,  the  Paladin,  the  NTCS-A,  the  NAVSTAR  GPS,  and  the 
REACT  systems.  In  performing  the  system  audits,  we  did  identify  and  report 
on  system-specific  deficiencies  in  the  areas  of  verifying  closure  of  action  items, 
updating  and  completing  interface  control  documents,  managing  waivers  and 
deviations,  updating  configuration  management  procedures,  and  performing 
type  certifications.  The  following  table  summarizes  the  system-specific  areas 
that  required  improvement  for  the  five  systems  reviewed. 
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Summary  of  Audit  Findings  for  the 
Five  Defense  Systems  Reviewed 


Areas  That  Required  Improvement 


Systems 
Reviewed  by 
Military 
Comoonent 

Verifying 
Closure  of 
Action 
Items 

Updating  and 
Completing 
Interface 
Control 
Documents 

Managing 
Waivers  and 
Deviations 

Updating 

Configuration 

Management 

Procedures 

Performing 

Type 

Certifications 

Army 

Kiowa  Warrior 

X 

X 

Paladin 

X 

Navy 

NTCS-A 

X 

Air  Force 

NAVSTAR  GPS 

X 

X 

REACT 

X 

X 

Each  program  office  took  appropriate  corrective  action  in  response  to  the 
system-specific  deficiencies.  In  addition,  we  suggested  that  the  NTCS-A 
Program  Office  submit  its  configuration  management  plan  to  the  Deputy  Under 
Secretary  of  Defense  (Acquisition  Reform)  to  be  included  in  the  Defense 
Acquisition  Deskbook  as  an  example  of  an  effective  way  to  tailor  the 
configuration  management  process  for  a  software  intensive  system  that  uses  an 
evolutionary  acquisition  strategy. 

Verifying  Closure  of  Action  Items.  The  Kiowa* Warrior  and  Paladin  Program 
Offices  needed  to  improve  internal  procedures  for  verifying  closure  of 
configuration-audit  action  items. 

Kiowa  Warrior  System.  The  Kiowa  Warrior  Program  Office  had  not: 

o  tracked  all  open  action  items  from  the  critical  design  review  of 
the  modifications  to  the  OH-58D  Kiowa  helicopter  to  validate  that  changes  were 
made  to  the  system  during  the  functional  and  physical  configuration  audits  and 

o  tracked  and  closed  out  all  action  items  from  each  functional 
and  physical  configuration  audit  before  it  certified  the  audits  as  complete. 

As  a  result,  the  Kiowa  Warrior  Program  Office  had  not  ensured  that  functional 
and  physical  configuration  audit  deficiencies  were  corrected  before  release  of 
the  design  for  Kiowa  Warrior  helicopter  production. 

Paladin  System.  The  Paladin  Program  Office  adequately  performed  the 
physical  configuration  audit  and  verified  the  closure  of  action  items.  However, 
for  the  functional  configuration  audits,  the  Program  Office  had  not  completed 
management  actions  needed  to  verify  the  closure  of  data-source-matrix  line 
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items  and  to  determine  whether  unverified  items  still  needed  to  undergo  a 
functional  configuration  audit.  The  Program  Office  considered  the  system- 
specification  requirements  to  be  configuration  items  and  defined  those 
configuration  items  as  data-source-matrix  line  items  for  verification  and  tracking 
purposes. 

Updating  and  Completing  Interface  Control  Documents.  The  NAVSTAR 
GPS  Joint  Program  Office  and  the  REACT  Program  Office  needed  to  place 
greater  emphasis  on  maintaining  current  and  complete  interface  control 
documents. 

NAVSTAR  GPS  System.  The  contractor  for  mission-processor 
software  used  incomplete  interface  control  documents  to  perform  formal 
qualification  tests  on  Block  HR  satellite  components.  The  NAVSTAR  GPS 
Joint  Program  Office  used  the  test  results  to  certify  the  successful  completion  of 
functional  configuration  audits  of  Block  IIR  satellite  components.  As  a  result, 
the  Joint  Program  Office  had  to  establish  an  operational  baseline  and  obligate  an 
additional  $15.6  million  to  modify  satellite  software,  conduct  additional  system 
tests,  and  establish  an  interface  control  agreement. 

REACT  System.  The  REACT  Program  Office  used  out-of-date 
interface  control  documents  to  perform  REACT  formal  qualification  tests.  The 
Program  Office  used  the  test  results  to  certify  the  successful  completion  of  the 
REACT  functional  configuration  audits.  For  example,  the  Program  Office  did 
not  verify  that  the  interface  control  documents  for  the  REACT  Program  were 
current  and  updated  for  the  formal  qualification  tests  of  the  REACT  voice 
control  panels  and  the  auxiliary  alarm  panel.  Consequently,  when  the 
contractor  conducted  the  formal  qualification  test,  the  test  results  were  based  on 
interface  control  documents  that  did  not  accurately  show  the  current  operational 
configuration  of  the  system.  As  a  result,  the  Program  Office  had  to  obligate  an 
additional  $1.1  million  to  redesign  and  retrofit  REACT  console  hardware 
configuration  items  and  had  to  conduct  another  functional  configuration  audit 
for  those  items. 

Managing  Waivers  and  Deviations.  The  NAVSTAR  GPS  Joint  Program 
Office  and  the  REACT  Program  Office  needed  to  improve  procedures  for 
reviewing,  identifying,  documenting,  and  obtaining  consideration  for 
contractor-requested  waivers  and  deviations. 

NAVSTAR  GPS  System.  For  11  of  the  24  waivers,  deviations,  and 
engineering  changes  reviewed,  the  NAVSTAR  GPS  Joint  Program  Office  did 
not  document  the  cost  reductions  or  other  consideration  that  were  appropriate 
and  did  not  obtain  such  consideration  from  the  contractor  for  6  of  the  11 
actions.  As  a  result,  the  procuring  contracting  officer  did  not  have  a  basis  for 
determining  whether  the  contractors  were  proposing  adequate  consideration  for 
the  waivers,  deviations,  and  engineering  changes.  Further,  the  Joint  Program 
Office  had  no  assurance  that  it  obtained  adequate  consideration  from  the 
contractors  for  granting  the  waivers,  deviations,  and  engineering  changes. 

REACT  System.  The  REACT  Program  Office  did  not  adequately 
document  its  technical  reviews  and  did  not  perform  cost  and  price  analyses  to 
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assess  the  adequacy  of  cost  reductions  and  other  consideration  offered  by  the 
contractor  for  waivers  of  contract  specifications.  As  a  result,  the  Program 
Office  had  no  assurance  that  contractors  compensated  the  Government 
adequately  for  eight  approved  waivers,  totaling  $439,000,  and  would 
compensate  the  Government  adequately  for  future  waivers  and  deviations. 

Updating  Configuration  Management  Procedures.  The  NTCS-A  Program 
Office  adequately  managed  the  functional  and  physical  configuration  audit 
process.  However,  the  NTCS-A  Program  Office  had  not  updated  its  procedures 
for  identifying,  documenting,  and  verifying  the  functional  and  physical 
characteristics  of  configuration  items  to  conform  with  current  practices. 
Specifically,  the  NTCS-A  Configuration  Management  Plan  that  contained  the 
procedures  did  not  show  the  tailored  configuration  control  process  used  for  the 
NTCS-A  Program,  which  is  a  software  intensive  system  that  uses  an 
evolutionary  acquisition  strategy. 

Performing  Type  Certifications.  The  Kiowa  Warrior  Program  Office 
accepted  the  Federal  Aviation  Administration's  performance  of  type  certification 
for  the  Kiowa  Warrior's  turboshaft  engine  in  place  of  Allison  Engine 
Company's  performance  of  the  contractually  required  functional  and  physical 
configuration  audits.  However,  that  Federal  Aviation  Administration 
certification  was  not  a  totally  adequate  replacement  for  the  configuration  audits 
because  the  Federal  Aviation  Administration  certification  process  did  not  ensure 
independent  DoD  oversight  to  verify  proper  functioning  and  design  of  the 
engine  against  Kiowa  Warrior  Program-unique  requirements.  As  a  result, 
without  independent  DoD  oversight,  the  Kiowa  Warrior  Program  cannot  be 
assured  that  the  engine  has  a  stable  product  baseline  and  that  product  baseline 
documentation  adequately  describes  all  Kiowa  Warrior  unique  functional  and 
physical  characteristics;  acceptance  testing  criteria;  and  tests  necessary  for 
deployment  and  installation,  logistics  support,  training,  and  disposal  of  the 
engine.  Further,  the  contractor  may  have  received  windfall  compensation  by 
not  being  required  to  perform  the  configuration  audits. 


Conclusion 

Overall,  the  Military  Departments  and  the  program  offices  for  the  five  Defense 
systems  included  in  our  review  were  adequately  managing  the  functional  and 
physical  configuration  audit  process.  We  commend  the  five  program  offices  for 
their  responsiveness  to  system-specific  recommendations  made  to  improve  the 
functional  and  physical  configuration  audit  process.  The  adequacy  of  the 
process  used  to  identify,  document,  and  verify  functional  and  physical 
characteristics  of  a  system  is  critical  to  ensuring  that  a  system  operates  properly 
and  can  be  supported  efficiently  and  effectively.  By  not  implementing  adequate 
processes,  the  systems  that  we  reviewed  spent  over  $16  million  to  correct 
problems  that  could  have  been  avoided;  potentially  paid  contractors  for  work 
not  done  in  accordance  with  contractual  requirements  by  not  determining  and 
obtaining  adequate  consideration;  and  put  the  systems  at  risk  of  future  problems 
and  changes  by  not  ensuring  stable  product  baselines. 
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Management  Comments 

We  provided  a  draft  of  this  report  to  you  on  October  30,  1996.  Because  the 
report  contains  no  findings  or  recommendations,  written  comments  were  not 
required,  and  none  were  received.  Therefore,  we  are  publishing  this  report  in 
final  form. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  If  you  have  questions 
on  this  report,  please  contact  Mr.  John  E.  Meling,  Audit  Program  Director,  at 
(703)  604-9091  (DSN  664-9091)  or  Mr.  Jack  D.  Snider,  Audit  Project 
Manager,  at  (703)  604-9087  (DSN  664-9087).  Enclosure  4  lists  the  distribution 
of  this  report.  The  audit  team  members  are  listed  inside  the  back  cover. 

David  K.  Steensma 
Deputy  Assistant  Inspector  General 
for  Auditing 


Enclosures 
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Scope  and  Methodology 


This  enclosure  discusses  the  scope  and  methodology  used  to  accomplish  the 
objective  as  well  as  organizations  and  individuals  visited  or  contacted, 
management  controls,  and  prior  audit  coverage. 


Scope 


We  conducted  this  audit  from  January  1995  through  October  1996  and  reviewed 
data  dated  from  October  1981  through  June  1996  relating  to  the  five  Defense 
systems  reviewed:  the  Army  Kiowa  Warrior  and  Paladin,  the  Navy  NTCS-A, 
and  the  Air  Force  NAVSTAR  GPS  and  REACT.  To  accomplish  the  objective, 
we  judgmentally  selected  the  systems  for  review.  As  a  result  of  our  review,  we 
issued  a  separate  report  for  each  system  and  synopsized  those  reports  in 
Enclosure  3.  We  used  the  results  of  those  reports  to  summarize  our  evaluation 
of  the  adequacy  of  the  functional  and  physical  configuration  audit  process  for 
the  acquisition  of  Defense  systems. 


Methodology 

We  conducted  this  program  audit  in  accordance  with  auditing  standards  issued 
by  the  Comptroller  General  of  the  United  States,  as  implemented  by  the 
Inspector  General,  DoD.  We  included  such  tests  of  management  controls  as  we 
deemed  necessary.  We  did  not  rely  on  computer-processed  data  to  develop 
conclusions  on  this  audit.  Technical  experts  from  the  Quantitative  Methods  and 
Technical  Assessment  Divisions  of  the  Analysis,  Planning,  and  Technical 
Support  Directorate,  Inspector  General,  DoD,  and  consultants  from  the 
Acquisition  Management  Directorate,  Inspector  General,  DoD,  assisted  in  the 
audit. 


Organizations  and  Individuals  Visited  or  Contacted 

We  visited  or  contacted  individuals  and  organizations  within  the  DoD  and  the 
Department  of  Transportation;  Serv-Air,  Blue  Grass  Depot  Activity;  Aerospace 
Corporation;  Allison  Engine  Company;  Bell  Helicopter  Textron,  Incorporated; 
Honeywell  Incorporated;  International  Telephone  and  Telegraph;  Lockheed 
Martin;  McDonnell  Douglas  Corporation;  TRW;  and  United  Defense,  Limited 
Partnership.  Further  details  are  available  on  request. 


Management  Control  Program 

Requirement  for  Management  Control  Review.  DoD  Directive  5010.38, 
"Internal  Management  Control  Program,"  April  14,  1987,*  requires  DoD 

+ 

DoD  Directive  5010.38  has  been  revised  as  "Management  Control  (MC)  Program,"  August  26, 
1996.  The  audit  was  performed  under  the  April  1987  version  of  the  directive. 

_  Enclosure  1 
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Scope  and  Methodology 


organizations  to  implement  a  comprehensive  system  of  management  controls 
that  provides  reasonable  assurance  that  programs  are  operating  as  intended  and 
to  evaluate  the  adequacy  of  the  management  controls. 

Scope  of  Review  of  Management  Control  Program.  We  limited  our  review 
because  of  relevant  coverage  in  Inspector  General,  DoD,  Report  No.  96-028, 
"Implementation  of  the  DoD  Management  Control  Program  for  Major  Defense 
Acquisition  Programs,"  November  28,  1995.  The  report  discusses  the 
effectiveness  of  the  management  control  program  that  the  Defense  Acquisition 
Executive  and  the  Component  Acquisition  Executives  used  for  major  Defense 
acquisition  programs.  The  report  concludes  that  the  acquisition  community  had 
not  effectively  integrated  DoD  Management  Control  Program  requirements  into 
its  management  assessment  and  reporting  processes.  As  a  result  of  the  report 
recommendations,  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  integrated  DoD  Directive  5010.38  requirements  into  the  March  15, 
1996,  revisions  to  DoD  Directive  5000.1,  "Defense  Acquisition,"  and  DoD 
Regulation  5000. 2-R.  Acquisition  managers  are  now  to  use  program  cost, 
schedule,  and  performance  parameters  as  control  objectives  to  implement  the 
DoD  Directive  5010.38  requirements.  The  managers  are  to  identify  material 
weaknesses  through  deviations  from  approved  acquisition  program  baselines  and 
exit  criteria  in  the  "Defense  Acquisition  Executive  Summary"  report. 
Accordingly,  we  limited  our  review  to  management  controls  over  the  functional 
and  physical  configuration  audit  process  at  the  Kiowa  Warrior,  Paladin, 
NTCS-A,  NAVSTAR  GPS,  and  REACT  Program  Offices. 

Adequacy  of  Management  Controls.  Management  controls  were  adequate  in 
that  we  did  not  identify  any  systemic  management  control  weakness  applicable 
to  the  audit  objective  for  the  Kiowa  Warrior,  Paladin,  NTCS-A,  NAVSTAR 
GPS,  and  REACT  systems. 


Prior  Audit  Coverage 

During  the  last  5  years,  the  General  Accounting  Office  and  the  Military 
Department  audit  agencies  have  not  issued  reports  addressing  the  functional  and 
physical  configuration  audit  process  or  the  application  of  the  process  in  the  five 
systems  included  in  this  audit.  Synopses  of  the  Office  of  the  Inspector  General, 
DoD,  reports  on  the  five  systems  reviewed  are  at  enclosure  3. 
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Definitions  of  Technical  Terms 


Action  Item.  A  document  that,  because  of  a  functional  or  physical 
configuration  audit,  requires  correction  of  a  deficiency  in  the  functional 
characteristics  or  technical  documentation  associated  with  a  configuration  item. 

Configuration  Control  Board.  A  Government  or  contractor  board  composed 
of  technical  and  administrative  representatives  who  recommend  approval  or 
disapproval  of  proposed  engineering  changes  to  a  configuration  item's  current 
approved  configuration  documentation.  The  board  also  recommends  approval 
or  disapproval  of  proposed  waivers  and  deviations  from  a  configuration  item's 
current  approved  configuration  documentation. 

Configuration  Identification.  The  process  of  establishing  and  describing  the 
contractual  baselines  and  related  configuration  items. 

Configuration  Item.  An  aggregation  of  hardware,  firmware,  or  computer 
software  or  any  of  their  discrete  portions  that  satisfies  an  end  use  function  and  is 
designated  by  the  Government  for  separate  configuration  management. 

Configuration  Management.  Technical  and  administrative  direction  and 
surveillance  actions  taken  to  identify  and  document  functional  and  physical 
characteristics  of  an  item,  to  control  changes  to  a  item  and  its  characteristics, 
and  to  record  and  report  change  processing  and  implementation  status. 

Configuration  Management  Plan.  A  document  defining  how  configuration 
management  will  be  implemented,  including  policies  and  procedures,  for  a 
particular  acquisition  or  program. 

Critical  Design  Review.  A  review  conducted  to: 

o  determine  that  the  detailed  design  satisfies  performance  and 
engineering  requirements  of  the  development  specification; 

o  establish  the  detailed  design  compatibility  requirements  of  the 
development  specification; 

o  establish  the  detailed  design  compatibility  among  the  item  and  other 
items  of  equipment,  facilities,  computer  program,  and  personnel; 

o  assess  producibility  and  risk  areas;  and 

o  review  the  preliminary  product  specifications. 

Data-Source-Matrix  Line  Item.  Configuration  item  derived  from  the  Paladin 
Program's  system  specification  requirements. 
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Deviation.  A  written  authorization,  granted  before  the  manufacture  of  an  item, 
to  depart  from  a  particular  performance  or  design  requirement  of  a 
specification,  drawing,  or  other  document  for  a  specific  number  of  units  or  a 
specified  period. 

Engineering  Change  Proposal.  A  contractor  document  that  describes  and 
justifies  a  proposed  engineering  change  and  applicable  costs  and  that  a 
contractor  submits  to  the  Government  for  approval  or  disapproval. 

Evolutionary  Acquisition.  An  acquisition  strategy  used  to  procure  a  system 
that  evolves  during  development  to  achieve  an  overall  system  capability.  An 
underlying  factor  in  evolutionary  acquisition  is  the  need  to  field  a  well-defined 
core  capability  quickly  in  response  to  a  validated  requirement,  while  planning 
through  an  incremental  upgrade  program  to  eventually  enhance  the  system  to 
provide  the  overall  system  capability. 

Firmware.  The  combination  of  a  hardware  device  and  computer  instructions  or 
computer  data  that  reside  as  read-only  software  on  the  hardware  device. 

Formal  Qualification  Test.  A  system  level  test  to  verify  that  the  configuration 
item  meets  the  performance  requirements  of  the  system  specification. 

Functional  Configuration  Audit.  A  formal  examination  of  functional 
characteristics  of  test  data  for  configuration  items  to  verify  that  the  items  have 
achieved  the  performance  specified  in  their  functional  or  allocated  identification. 
If  the  items  were  developed  at  Government  expense,  the  functional 
configuration  audit  must  be  performed  before  acceptance  of  the  items.  The 
functional  configuration  audit  must  be  performed  on  a  prototype  or  the 
configuration  to  be  released  for  production  of  the  operational  quantities. 

Interface  Control  Document.  A  technical  agreement  required  to  successfully 
develop  interoperable  configuration  items  designed  independently  by  technical 
engineers. 

Low-Rate  Initial  Production.  The  production  of  a  system  in  limited  quantity 
to  provide  articles  for  operational  test  and  evaluation  and  to  establish  an  initial 
production  rate  sufficient  to  lead  to  full-rate  production  upon  successful 
completion  of  operational  testing. 

Mission-Processor  Software.  Software  that  allows  the  NAVSTAR  GPS 
satellite  to  process  and  transmit  navigational  data  to  and  from  the  master  ground 
control  station. 

Operational  Baseline.  The  product  baseline  as  updated  with  subsequent 
software  modifications  to  make  the  product  baseline  satisfy  operational 
requirements  for  current  and  future  satellite  operations. 

Physical  Configuration  Audit.  A  formal  examination  to  verify  that  the 
configuration  item  "as  built"  conforms  to  the  technical  documentation  that 
defines  the  item.  The  physical  configuration  audit  includes  a  detailed  audit  of 
engineering  drawings,  specifications,  technical  data,  and  tests  used  in 
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production  of  the  item.  The  physical  configuration  audit  may  be  conducted  on 
the  first  full-rate  production  or  the  first  low-rate  initial  production  item. 
Government  program  office  approval  of  the  product  specification  and 
satisfactory  completion  of  the  physical  configuration  audit  establish  the  product 
baseline.  A  contractor  is  required  to  process  all  future  changes  to  the  product 
baseline  by  the  formal  engineering  change  proposal  process. 

Product  Baseline.  The  baseline  established  at  the  physical  configuration  audit 
that  includes  product,  process,  and  material  specifications  and  engineering 
drawings.  Approval  of  the  configuration  item  product  specification  by  the 
Government  program  office  and  satisfactory  completion  of  the  physical 
configuration  audit  establish  the  product  baseline. 

Prototype.  An  original  or  model  on  which  a  later  item  is  formed  or  based. 

Rework.  Any  corrections  of  defective  work  either  before,  during,  or  after 
inspection. 

Specification.  A  document  intended  primarily  for  use  in  procurement  that 
clearly  and  accurately  describes  the  essential  technical  requirements  for  items, 
materials,  or  services,  including  the  procedures  for  determining  whether  the 
requirements  have  been  met. 

Waiver.  A  written  authorization  to  accept  a  configuration  item  that  departs 
from  specified  requirements.  The  item  may  be  considered  suitable  "as  is"  or 
after  rework  by  an  approved  method. 
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From  January  through  October  1996,  the  Inspector  General,  DoD,  issued  five 
audit  reports  involving  Defense  systems  with  issues  related  to  the  functional  and 
physical  configuration  audit  process. 

Audit  Report  No.  97-011,  "Functional  and  Physical  Configuration  Audits  of 
the  Air  Force  Navigation  System  Time  and  Ranging  Global  Positioning 
System,"  October  28,  1996.  The  report  states  that,  overall,  the  functional  and 
physical  configuration  audit  processes  for  the  NAVSTAR  GPS  were  well 
managed;  however,  two  areas  needed  improvement. 

o  The  contractor  for  mission-processor  software  used  incomplete 
interface  control  documents  to  perform  formal  qualification  tests  on  Block  HR 
satellite  components.  The  NAVSTAR  GPS  Joint  Program  Office  used  those  test 
results  to  certify  the  successful  completion  of  functional  configuration  audits  of 
Block  HR  satellite  components.  As  a  result,  the  Joint  Program  Office  had  to 
establish  an  operational  baseline  and  obligate  an  additional  $15.6  million  to 
modify  satellite  software,  conduct  additional  system  tests,  and  establish  an 
interface  control  agreement. 

o  For  11  of  the  24  waivers,  deviations,  and  engineering  changes 
reviewed,  the  Joint  Program  Office  did  not  document  the  cost  reductions  or 
other  consideration  that  were  appropriate  and  did  not  obtain  such  consideration 
from  the  contractor  for  6  of  die  11  actions.  As  a  result,  the  Joint  Program 
Office  had  no  assurance  that  it  received  adequate  consideration  for  waivers  and 
deviations  approved  for  the  miniaturized  airborne  NAVSTAR  GPS  receivers; 
for  engineering  changes  approved  for  the  Block  HR  satellite;  and  for  future 
waivers,  deviations,  and  engineering  changes. 

We  recommended  that  the  NAVSTAR  GPS  System  Program  Director  require 
approval  and  certification  of  complete  interface  control  documents  before  future 
critical  design  reviews  and  formal  qualification  tests,  designate  the  interface 
control  process  as  a  high  risk  management  control  function  in  the  management 
control  program,  and  require  NAVSTAR  GPS  project  officers  and  the  Joint 
System  Configuration  Control  Board  to  document  their  recommendations  for 
equitable  contract  price  adjustments  or  other  consideration  for  pending  and 
future  waivers,  deviations,  and  engineering  changes.  We  also  recommended 
that  the  Director  of  Contracting,  NAVSTAR  GPS,  modify  applicable  contracts 
for  approved  pending  and  future  waivers,  deviations,  and  engineering  changes 
to  obtain  an  equitable  price  adjustment  or  other  consideration  after  considering 
recommendations  from  the  NAVSTAR  GPS  Joint  Program  Office. 

In  response  to  the  report,  the  System  Program  Director  concurred  with  the 
recommendations  concerning  approval  and  certification  of  complete  interface 
control  documents,  designation  of  the  interface  control  process  as  a  high  risk 
management  control  function,  and  documentation  of  project  officer  and  Joint 
System  Configuration  Control  Board  recommendations.  In  response  to  the 
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recommendation  concerning  the  modification  of  applicable  contracts  for 
waivers,  deviations,  and  engineering  changes,  the  Deputy  Director  of 
Contracting,  NAVSTAR  GPS,  concurred  with  the  recommendation  and 
provided  actions  taken  and  planned. 

Audit  Report  No.  96-211,  "Functional  and  Physical  Configuration  Audits  of 
the  Navy  Tactical  Command  System- Afloat,"  August  14,  1996.  The  report 
states  that  the  NTCS-A  Program  Office  adequately  managed  the  functional  and 
physical  configuration  audit  process.  However,  the  NTCS-A  Program  Office 
had  not  updated  its  procedures  for  identifying,  documenting,  and  verifying  the 
functional  and  physical  characteristics  of  configuration  items  to  conform  with 
current  practices.  During  our  audit,  the  NTCS-A  Program  Office  began 
updating  its  configuration  management  plan  to  incorporate  current  functional 
and  physical  configuration  management  procedures.  The  report  did  not  contain 
any  recommendations  because  the  Program  Office  took  corrective  action  during 
the  audit. 

Audit  Report  No.  96-130,  "Functional  and  Physical  Configuration  Audits  of 
the  Army  Paladin  Program,"  May  24,  1996.  The  report  states  that  the 
Paladin  Program  Office  adequately  performed  its  physical  configuration  audit 
and  verified  the  closure  of  action  items.  However,  for  the  functional 
configuration  audits,  the  Program  Office  had  not  completed  management  actions 
needed  to  verify  the  closure  of  data-source-matrix  line  items  and  to  determine 
whether  unverified  items  still  needed  to  undergo  a  functional  configuration 
audit.  During  our  audit,  the  Program  Office  took  appropriate  corrective  actions 
to  verify  and  close  the  items  and  to  conduct  functional  configuration  audits  of 
unverified  items.  The  report  did  not  contain  any  recommendations  because  the 
Program  Office  took  corrective  actions  during  the  audit. 

Audit  Report  No.  96-073,  "Functional  and  Physical  Configuration  Audits  of 
the  Air  Force  Rapid  Execution  and  Combat  Targeting  Program," 
February  16,  1996.  The  report  states  that  the  functional  and  physical 
configuration  audit  processes  for  the  REACT  Program  needed  improvement. 

o  The  REACT  Program  Office  used  out-of-date  interface  control 
documents  to  perform  REACT  formal  qualification  tests.  The  Program  Office 
used  the  test  results  to  certify  the  successful  completion  of  the  REACT 
functional  configuration  audits.  As  a  result,  the  Program  Office  had  to  obligate 
an  additional  $1.1  million  to  redesign  and  retrofit  affected  REACT  console 
hardware  items  and  had  to  conduct  another  functional  configuration  audit  for 
those  items. 

o  The  REACT  Program  Office  did  not  adequately  document  its 
technical  reviews  and  did  not  perform  cost  and  price  analyses  to  assess  the 
adequacy  of  cost  reductions  and  other  consideration  offered  by  the  contractor 
for  waivers  of  contract  specifications.  As  a  result,  the  Program  Office  had  no 
assurance  that  contractors  compensated  the  Government  adequately  for 
eight  waivers,  totaling  $439,000,  and  would  compensate  the  Government 
adequately  for  future  waivers  and  deviations. 
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We  recommended  that  the  REACT  Program  Manager  assign  a  higher  priority  to 
maintaining  current  interface  control  documents  and  obtain  equitable  price 
adjustments  or  other  consideration  in  return  for  approving  waivers  of 
specifications  based  on  documented  cost  and  price  analyses.  In  response  to  the 
report,  the  Program  Executive  Officer,  Bombers,  Missiles  and  Trainers, 
concurred  with  the  recommendations  and  took  appropriate  action  to  implement 
the  recommendations. 

Audit  Report  No.  96-063,  "Functional  and  Physical  Configuration  Audits  of 
the  Army  Kiowa  Warrior  Program,"  January  31,  1996.  The  report  states 
that  the  functional  and  physical  configuration  audit  process  for  the  Kiowa 
Warrior  Program  needed  improvement. 

o  The  Kiowa  Warrior  Program  Office  did  not  adequately  manage  the 
functional  and  physical  configuration  audit  process  for  the  Kiowa  Warrior 
Program.  As  a  result,  the  Program  Office  may  incur  additional  costs  from 
reverse  engineering,  redesign,  and  retrofit  during  production. 

o  The  Kiowa  Warrior  Program  Office  accepted  the  Federal  Aviation 
Administration's  performance  of  type  certification  for  the  Kiowa  Warrior's 
turboshaft  engine  in  place  of  Allison  Engine  Company's  performance  of  the 
contractually  required  functional  and  physical  configuration  audits.  As  a  result, 
the  Kiowa  Warrior  Program  cannot  be  assured  that  the  engine  has  a  stable 
product  baseline  and  that  product  baseline  documentation  adequately  describes 
all  Kiowa  Warrior  unique  functional  and  physical  characteristics,  acceptance 
testing  criteria,  and  post-production  testing.  Further,  the  contractor  may  have 
received  windfall  compensation  by  not  being  required  to  perform  the 
configuration  audits. 

We  recommended  that  the  Kiowa  Warrior  Program  Office  complete  the  Kiowa 
Warrior  helicopter  product  baseline,  verify  closure  status  of  configuration  audit 
action  items,  obtain  complete  drawings  and  technical  data  packages,  designate  a 
DoD  representative  to  supplement  the  Federal  Aviation  Administration  engine 
inspection  and  certification  process,  and  correct  engine  problems  before 
awarding  future  engine  production  contracts. 

In  response  to  the  report,  the  Action  Officer,  Office  of  the  Assistant  Secretary 
of  the  Army  (Research,  Development  and  Acquisition),  did  not  concur  with  all 
of  our  recommendations;  however,  he  proposed  alternative  actions  that  met  the 
intent  of  our  recommendations.  The  following  represents  the  responses  to  our 
recommendations . 

o  Concerning  the  completion  of  the  Kiowa  Warrior  helicopter  product 
baseline,  the  Action  Officer  stated  that  completing  the  product  baseline  would 
not  be  practical;  however,  he  stated  that  the  Kiowa  Warrior  Program  Office  is 
conducting  functional  and  physical  configuration  audits  for  new  system 
modifications  that  will  ensure  that  the  configuration  audit  process  is  in  place,  is 
functioning,  and  meets  the  needs  of  good  program  management. 
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o  Concerning  the  verification  of  the  closure  status  of  configuration  audit 
action  items,  the  Action  Officer  stated  that  the  Kiowa  Warrior  Program  Office 
implemented  a  tracking  and  closure  process  for  open  action  items  resulting  from 
functional  and  physical  configuration  audits. 

o  Concerning  the  obtaining  of  complete  drawings  and  technical  data 
packages,  the  Action  Officer  stated  that  the  Kiowa  Warrior  Program  Manager 
should  specifically  evaluate  the  need  to  acquire  complete  drawings  and  technical 
data  packages  for  possible  future  second  sourcing  or  competitive  bids. 

o  Concerning  the  designation  of  a  DoD  representative  to  supplement  the 
Federal  Aviation  Administration  engine  inspection  and  certification  process,  the 
Action  Officer  stated  that  current  procedures  adequately  cover  certification 
requirements,  including  those  of  the  Federal  Aviation  Administration  and 
military.  However,  he  stated  that  the  Army  engine  airworthiness  review  for  the 
250-C30R/3  engine  would  consider  Federal  Aviation  Administration  and 
military  qualification  data. 

o  Concerning  the  correction  of  engine  problems  before  awarding  future 
engine  production  contracts,  the  Action  Officer  stated  that  the  production 
approval  for  the  250-C30R/3  engine  would  not  be  given  until  the  Army 
airworthiness  review  had  been  completed. 
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